1. Field of the Invention
The present invention relates to an image processing method and apparatus, which generate and output an image at the viewpoint position of the observer on the basis of a plurality of input images at different viewpoints.
2. Related Background Art
In a conventional method, in order to generate an image observed at a position other than the viewpoint position of the image from a plurality of images, the three-dimensional shape or the like of an object is temporarily reconstructed using, e.g., stereoscopic distance measurements, and an image observed from the new viewpoint position is generated using the reconstructed shape.
However, it is generally difficult to calculate the three-dimensional shape of an object from actually taken images. If the object has a simple shape, its shape may be reconstructed by calculating corresponding points by, e.g., stereoscopic distance measurements, but it is nearly impossible for natural objects such as trees having complex shapes to attain such reconstruction. Therefore, the method of calculating the three-dimensional shape and generating an image at a new viewpoint position based on the calculated shape cannot be used for such natural objects.
It is, therefore, an object of the present invention to provide an image processing method and apparatus, which generate an image at an arbitrary viewpoint position, which suffers less distortion, from images taken at a plurality of viewpoint positions.
It is another object of the present invention to provide an image processing method and apparatus, which can assure a broad observation range that allows movement of the viewpoint, and can suppress an increase in required data volume even in such case.
In order to achieve the above objects, the present invention comprises:
image input means for inputting images of an object taken at a plurality of viewpoint positions together with information indicating the plurality of viewpoint positions;
viewpoint position detection means for detecting a viewpoint position of an observer;
selection means for selecting an image from the images input by the image input means on the basis of the viewpoint position detected by the viewpoint position detection means;
image generation means for generating an image at the viewpoint position detected by the viewpoint position detection means on the basis of the image selected by the selection means in accordance with the viewpoint position detected by the viewpoint position detection means and the viewpoint position upon taking the image selected by the selection means; and
image display means for displaying the image generated by the image generation means.
Also, the present invention comprises:
the image input step of inputting images of an object taken at a plurality of viewpoint positions together with information indicating the plurality of viewpoint positions;
the viewpoint position detection step of detecting a viewpoint position of an observer;
the selection step of selecting an image from the images input in the image input step on the basis of the viewpoint position detected in the viewpoint position detection step;
the image generation step of generating an image at the viewpoint position detected in the viewpoint position detection step on the basis of the image selected in the selection step in accordance with the viewpoint position detected in the viewpoint position detection step and the viewpoint position upon taking the image selected in the selection step; and
the image display step of displaying the image generated in the image generation step.
Furthermore, the present invention comprises:
multi-viewpoint image input means for inputting a plurality of images having different viewpoint positions as multi-viewpoint image data;
image conversion means for converting the input multi-viewpoint image data into light rays that travel in a space;
light ray recording means for recording each of the converted light rays as light space data on a light space corresponding to a plane which is one of a plurality of radially arranged reference planes, and has an angle the plane makes with the light ray, which is closest to right angles;
viewpoint position and direction of line of sight input means for inputting a viewpoint position and a direction of line of sight of an image to be generated;
image calculation means for calculating an image required for determining each pixel value from the light space data in units of pixels of the image to be generated;
pixel position calculation means for calculating a position of a required scan line and a pixel position in the scan line in the light space data;
pixel value determination means for determining each pixel value of the image to be generated;
image generation means for generating a desired image on the basis of the calculated image, the pixel positions, and the pixel values; and
display means for displaying the generated image on a display screen.
Moreover, the present invention comprises:
the multi-viewpoint image input step of inputting a plurality of images having different viewpoint positions as multi-viewpoint image data;
the image conversion step of converting the input multi-viewpoint image data into light rays that travel in a space;
the light ray recording step of recording each of the converted light rays as light space data on a light space corresponding to a plane which is one of a plurality of radially arranged reference planes, and has an angle the plane makes with the light ray, which is closest to right angles;
the viewpoint position and direction of line of sight input step of inputting a viewpoint position and a direction of line of sight of an image to be generated;
the image calculation step of calculating an image required for determining each pixel value from the light space data in units of pixels of the image to be generated;
the pixel position calculation step of calculating a position of a required scan line and a pixel position in the scan line in the light space data;
the pixel value determination step of determining each pixel value of the image to be generated;
the image generation step of generating a desired image on the basis of the calculated image, the pixel positions, and the pixel values; and
the display step of displaying the generated image on a display screen.